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Article history: This paper presents the evaluation of positive emotion in children's mobile 
learning applications. The mobile learning application is a teaching aid that 
Received Oct 31, 2019 can help students to self-study and increase the students’ interest in learning 
Revised Dec 28, 2019 especially children. This paper will discuss how mobile learning application 
Accepted Feb 7, 2020 affects the children interest in school. The evaluation method implemented 
to evaluate the rate of positive emotion elicited by the children using mobile 
learning applications was a mixed method of qualitative and quantitative 
methods. Since emotion can be either negative or positive, the identification 
: of a proper method or perspective was required to prove that positive 
Children emotion was really elicited. Next, the data was collected through 
the children’s assessment score, Electroencephalograms (EEG) device, 
Emotion identification using micro-expression (facial expression), Kort Scale 
Positive emotion and interview to confirm the positive emotion felt by the students. The result 
shows that all five perspectives or methods have shown that positive emotion 
is produced. It is found that the M obile learning application can really trigger 

the children’s positive emotions. 


Keywords: 


Methodology 
Mobile learning application 


This is an open access article under the CC BY-SA license. 


Corresponding Author: 


Aslina Baharum, 

Faculty of Computing and Informatics, Universiti Malaysia Sabah, 
88400 Kota Kinabalu, Sabah, Malaysia. 

Email: aslina@ums.edu.my 


1. INTRODUCTION 

In study on human behaviour it has been suggested that creating positive associations in learning 
was more effective than the negative ones [1]. The positive emotion felt by the students are beneficial 
in encountering student’s academic challenges. Moreover, the positive emotion can be implemented 
in mobile learning application to spark the emotional engagement among students [2]. Besides, there is 
a clear consensus in the psychological and educational fields that a variety of positive emotions play critical 
roles in fostering learning and mental growth in children [3-4]. Mobile technologies have become an integral 
part of people's daily life that showed a significant progress in its development within recent years [5]. 
Nowadays, the m-learning (Mobile Learning) has become the attention and focus of several scientists 
and work on its development and distribution fromall over the world [6-8]. The mobile learning term denotes 
the use of mobile applications installed on a mobile device (tablet or smartphone) that used in the education 
process at school [9-10]. The implementation of this mobile learning is not a simple process when it is 
for young age students and children’s learning. Textbook’s author, children (students), parents and teachers 
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play an important role in the formation of the children character, interests and social behaviour through 
learnings’ process [11]. Therefore, by applying the M-learning in children education, it’s shown a significant 
contribution for children to build their character, behaviour and interest in leaming. This could be done 
by developing an e-learning application that could run on an Android device platform. 

According to Desmet (2007), it was possible to design products that target specific types 
of emotions by measuring emotional responses [12]. The products that excite user will be more successful 
than those that do not has been supported by consumers and researchers [13]. According to Ashby & Johnson 
(2013), exploration of other senses or emotional connection, associations and experience were important for 
the product to compete [14]. The link between attachment and emotional design was important for the user to 
use the product [15]. During the development of the product, the contrasting and categorization of emotions 
describes how the emotions are correlated to each other. According to Barrett (2006), humans experience 
emotion through evidence that influence action, thoughts and behaviour. 

The research also suggested that emotion can be categorized into various affect related to the human 
current situation [16]. Next, the emotions can be categorized by identifying the most basic level of human 
emotion [17]. This was then supported by Robinson (2008), that identifying the basic emotion and its 
definition was required to the classification of emotion into positive and negative emotion [18]. This method 
was already used in researches such as [19-20]. Unfortunately, classifying the emotions into positive 
and negative emotion was not easy if the basic level of human emotion was not identified. Therefore, before 
the emotion can be evaluated, researcher need to understand what kind of emotion that can be considered 
positive and negative. 

Students that prone to positive emotions may experience high levels of achievement partly because 
such emotions are associated with high-quality peer and teacher relationships [21]. By adapting the positive 
emotion in learning, this process may improve student learning, achievement, and self-efficacy [22]. 
This prove that the positive emotion is directly proportional to the student achievement score. Other than that, 
micro-expressions (facial expression) contain a significant and effective amount of information regarding 
the true emotions which useful in practical applications such as security and interrogations [23]. This 
micro-expression (facial expression) was used to identify the children true emotion whether positive 
or negative emotion. The basic set of emotions is largely discussed but is not defined completely. (Kort et al., 
2002) constructed a scale of emotions related to learning ranging from negative to positive states (from -1 
to +1) that was used in this research to scale the emotion felt by the children [24]. In addition, 
Electroencephalograms (EEG) record the electric activity of a human's brain through electrodes that are 
attached to the scalp. According to (Ramirez, 2012) this device could detect and classify the emotion signal 
from brain wave with high accuracy [25]. Now, there are plenty of EEG device already available in market 
to detect the human emotion. 

Therefore, the evaluation of positive emotion in children mobile learning application help 
researchers to understand whether there are positives emotion triggered while the children using 
the application or not. The method used to evaluate such uncertainty was mix method of qualitative 
and quantitative method. The result of the evaluation helps the researchers and teachers to distinguish 
a succes sfullearning application that can be used among the children. 


2. RESEARCH METHOD 

A mobile learning application (Sains Awal PraSekolah) that already been developed was used in this 
research and an appropriate questionnaire was created. The mobile learning application was evaluated using 
three (5) different method that can express the participant’s real positive emotion after the participant used 
the application. In order to achieve that all aspects need to be considered like the effectiveness of the mobile 
learning application in giving the participant’s information such as the participant’s assessment score, 
the participant facial expression and their emotions. Moreover, the evaluation continued using 
electroencephalogram (EEG) device, evaluation using universal facial expression and evaluation using Kort’s 
Scale and interview on participant emotion after using the mobile learning application. After the data have 
been collected it then being analyzed. The data collection flow of this study was as follow see Figure 1. 

Figure 1 show the research method on how the evaluation of the positive emotion have been 
conducted. Firstly, a Mobile learning application must be selected that act as an object that being tested 
whether it can produce a positive or negative emotion to the participant. Next, a questionnaire prepared that 
according to the evaluation test in the data collection. Finally, the data that have been collected are analyzed 
according to its frequency and mean. The result of the data analysis shown in the next section. 
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Identify the Mobile learning Data Collection: 
application 1. Assessment Score 
2. EEG Device 
(Sains Awal PraSekolah) 3. Micro-Expression (Facial expression) 
4. Kort Scale 
5. Interview 


Note: 


1. High assessment score indicate positive 
Prepare Evaluation emotion (Valient, 2012) 
Questionnaire 
2. EEG device Identify emotion in high accuracy 
(Ramirez, 2012) 


3. Micro-expression could identify human true 
emotion (Frank et. al., 2009) 


Data Analysis and result of 
the evaluated mobile learning 4. Kort scale used to rate student emotion from 
application (+2 until -2) (Kort et. al, 2002) 


5. Interview the children emotion directly 
(Robson, 2016) 





Figure 1. Methodology ofthe positive emotion evaluation 


3. RESULTS AND ANALYSIS 

The evaluation of the mobile learning application was conducted in three different locations based 
on its sociology and economy. This to confirm whether that the mobile learning application could be used 
in a different level of technology exposure or not. The evaluation was conducted at a different location 
in Sabah that includes rural school (Tadika Advent Goshen Kota Marudu), suburban school (Pacos Trus 
Penampang) and urban school (Pusat Minda Lestari UMS Kota Kinabalu). There is a total of 104 participants 
that involve in the evaluation that consists of 98 children Table 1 and six teachers Table 2. 


Table 1. The demography of datacollected for children 


. Frequency 
Question! - Range Rural Siania Urban Total Porcenta: 
Male 25 12 15 52 53.1 
Gender Female 22 10 14 46 46.9 
Total 47 22 29 98 100 


Table 2. The demography of data collected for teacher 


: Frequency 
oa oe Rural Suburban Urban Total Perera 
Female 2 2 2 6 100 
ee ee 2 2 6 100 


3.1. Assessment score 

The assessment score is the quiz mark answered by the children in the e-learning application. 
The quiz aim is to test the children understanding the topic taught and the score is in percentage value. 
Three of the highest score in the quiz is recorded according to location and gender as shown in Table 3, 
Table 4 and Table 5. 


Table 3. Mean score of assessment for children at ““Tadika Advent Goshen” 


Participants Questions N Minimum Maximum Mean _ Std. Deviation 
Mark | (%) 47 25.00 100.00 93.94 14.07 
Rural Children Mark 2 (%) 47 50.00 100.00 93.79 13.04 
Mark 3 (%) 47 75.00 100.00 97.36 7.07 
Total Mean 50.00 100.00 95.03 11.39 


The mean scores of assessments for rural children are presented in Table 3. The result showed that 
the mark of three questions was recorded in percentage. This to ensure that the data collected was accurate 
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since it’s involving the children assessment score. The average minimum mark scored for all the question 
was 50.0% and the highest was 100.0%. This indicated that there were still a few children that do not perform 
well even though using the mobile application because the minimum mark scored was as low as 25.0%. 
In addition, the total average mean scored by the rural children was 95.0% with the standard deviation 
of 11.39. This shown that, most of the children still can perform well with such standard deviation. 

The mean scores of assessments for suburban children are shown as Table 4. The result showed that 
the average minimum mark scored for all the question was 60.0% and the highest was 100.0%. This indicated 
that the children level of understanding was moderate and some of them can perform well using the mobile 
application. In addition, the total average mean scored by the suburban children was 95.4% with the standard 
deviation of 11.54. This result shown that, the suburban children can perform almost the same with the rural 
children despite the children level of technology exposure. 


Table 4. Mean score of assessment for children at “Pacos Trust” 


“Participants Questions =N Minimum Maximum Mean Std Deviation — 
Saban Mark I (%) 22. 50.00 100.00 95.91 11.82 
Children Mark 2 (%) 22 80.00 100.00 96.50 7.62 

Mark 3 (%) 22 50.00 100.00 93.77 15.17 
Total Mean 60.00 100.00 95.39 11.54 


The mean scores of assessments for urban area children are presented in Table 5. The results 
showed that the average minimum mark scored for all the question was 71.7% and the highest was 100.0%. 
This indicated that the children level of understanding was almost excellent and can perform well using 
the mobile application. In addition, the total average mean scored by the urban children was 97.2% with 
the standard deviation of 7.48. This result shown that, the urban children can perform very well using 
the application. This prove that the children familiarity with the current technology enable the children 
to perform very well with the assessment score evaluation. The assessment score shows that the average 
of children can score 95.0% in rural area, 95.4% for the suburban area and 97.2% in urban area. This shows 
that the urban children can score more than the children in rural and in suburban area. Moreover, the average 
of those 3 areas can score about 95.9% with 10.14 standard deviations. This proves that the children can 
understand the learning materials given and can perform well with the quiz. This also prove that the mobile 
application can produce a positive experience to children almost 95.0%. 


Table 5. Mean score of assessment for children at “PML UMS” 


Participants Questions N Minimum Maximum Mean Std. Deviation 
Mark | (%) 29 75.00 100.00 95.28 9.53 
Urban Children Mark 2 (%) 29 60.00 100.00 97.07 9.21 
Mark 3 (%) 29 80.00 100.00 99.31 3.71 
Total Mean 71.67 100.00 97.22 7.48 


3.2. Hectroencephalograms device 

The Electroencephalogram (EEG) device capable of recording the brain activity in real time 
and shows the learning effectiveness. According to [26], the positive emotion felt by the student is directly 
proportional to the level of learning effectiveness. Therefore, the EEG device could be used as an evaluation 
method to identify the level of positive emotion felt by the children. According to Gasah (2018), children 
focus, and attention are only between three to five minutes [27]. Since the average of children attention only 
between three to five minutes, the data between that period only, will be considered for evaluation because 
children were in their full attention during that period. Table 6, Table 7 and Table 8 shows the result 
of the EEG. The percentage value represents the positive and negative emotion felt by the children less than 
fifty per cent (<50%) means negative emotion and more than fifty per cent (>50%) means positive emotion. 

Table 6 shows the EEG device score for rural children. The score of the EEG device for less than 
3 minutes indicate that almost 51.5% of effectiveness in children from 7 children only. The mean increases 
to 53.2% when the period was less than 5 minutes taken from 25 children. This result contradicts 
from the expected result because the effectiveness of learning should be decreased when the time increases. 

Table 7 shows the EEG device score for suburban children. The score of the EEG device for less 
than 3 minutes indicate that almost 82.2% of effectiveness in children from seven children. This then shows 
a significantly decreasing to 74.4% when the period is less than 5 minutes were taken from 10 children. 
This result followed the prediction of the result in phase one where the effectiveness in learning will be 
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decreasing to time. Even though, the level of high level of effectiveness that reaches more than 82.0% show 
that the mobile learning application in this area could build a positive emotion to children. 


Table 6. EEG device score at ““Tadika Advent Goshen” 


Rural Kids (Kota Marudu) 
Minute N Minimum Maximum Mean Std. Deviation 
<3 Mins Time (mm:ss) 7 1:38 2:55 2:22 0.02 
Percentage (%) 7 24.2 67.2 51.357 16.52 
Valid N (list wise) 7 
<5 Mins Time (mm:ss) 25 1:38 4:47 3:24 0.03 
Percentage (%) 25 24.2 95 53.208 19.68 


Valid N (listwise) 25 


Table 7. EEG device score at “Pacos Trust” 
Suburban Kids (Penampang) 


Minute N Minimum Maximum Mean Std. Deviation 
<3 Mins Time (mm:ss) 7 0:36 2:51 1:53 0.04 
Percentage (%) 7 68.3 100 82.186 11.18 
Valid N (list wise) 7 
<5 Mins Time (mm:ss) 10 0:36 4:35 2:35 0.06 
Percentage (%) 10 39.8 100 74.41 18.13 


Valid N (listwise) 10 


Table 8 shows the EEG device score for suburban children. The score of the EEG device for less 
than 3 minutes indicate that almost 78.7% of effectiveness in children were taken from three children only. 
This then shows a significantly decreasing to 61.752% when the period is less than 5 minutes were taken 
from 21 children. This result followed the prediction of the result in phase one where the effectiveness 
in learning will be decreasing to time. Even though, the level of high level of effectiveness that reaches more 
than 78.0% show that the mobile learning application in this area could build a positive emotion to children. 
The EEG device score shows that the average percentage of children having positive emotion in less than 
three-minute study for rural children (51.4%), suburban children (82.2%) and urban children (78.7%). 
This show that the e-learning application is more effective in creating a positive emotion in suburban children 
compared to urban children and rural children. Next, the study duration that more than three minute and less 
than five minutes show an increment for rural children (53.2%) and decrement to suburban children (74.4%) 
and urban children (61.8%) respectively. This proves the findings in phase | that the children learning 
efficiency will decrease within time because of two out of three schools shown the effectiveness decrement. 
Finally, the average of the 3 areas could EEG score for less than three minutes is 70.7% with standard 
deviation of 13.087 and less than five minute is 63.1% with standard deviation of 19.444. The result of 
the EEG score also shown that the application could create more positive emotion than the negative emotion. 


Table 8. EEG device score at ““ PML UMS” 
Urban Kids (Kota Kinabalu) 


Minute N Minimum Maximum Mean Std. Deviation 
<3 Mins Time (mm:ss) 3 2:06 2:59 2:27 0.02 
Percentage (%) 3 71.4 92 78.667 11.56 
Valid N (list wise) 3 
<5 Mins Time (mm:ss) 21 2:06 4:56 3:59 0.03 
Percentage (%) 21 29.9 94 61.752 20.52 


Valid N (listwise) 21 


3.3. Micro-expression 

The facial expression is also known as the micro expression. According to Ekman (2004) it is 
difficult to fake a micro expression because it is an involuntary facial expression that appears on a person’s 
face according to the emotions being experienced [28]. There is consist of 7 universal micro-expression: 
sadness, anger, fear, disgust, contempt, surprise and happy. Therefore, by identifying the facial expression 
of the children, their emotion could be identified. Figure 2, Figure 3 and Figure 4 show the result of the facial 
expression and its emotion. The result sorted based on gender and the presence of a facial expression that 
shows emotion. Figure 2 shows the facial expression result of rural children. The result of the micro- 
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expression (facial expression) graph indicated that most of the rural children leading towards happiness 
(19 children) and surprise (18 children). This facial expression proved that the children produce more 
positive emotion in facial expression compared to negative emotion. 

Figure 3 shows the facial expression result of suburban children. The result of the micro-expression 
(facial expression) graph indicated that most of the suburban children was happy (13 children) and surprise 
(11 children). This proves that the suburban children were mostly happy and surprise that indicated positive 
emotion when using the mobile application. 


Facial Expression Result for Rural Children Facial Expression Result for Suburban 
Happiness Children 
20 19 Happiness 
15 15-13 
Surprise 
10 Sadness 10 Surprise 
18 11 
—Rural Children 25 = Suburban Children 
o—* 3 
Contempt Anger Contempt 
3 
0 
Fear Disgust 





Figure 2. The facial expression result of rural children _—‘ Figure 3. The facial expression result of suburban 
children 


Figure 4 shows the facial expression result of urban children. The result of the micro-expression 
(facial expression) graph indicated that most of the urban children leading towards happiness (14 children) 
and surprise (12 children). This facial expression proved that the urban children produce more positive 
emotion in facial expression compared to negative emotion. The graph for all of the area shows 
a significantly high frequency towards the positive emotion (Happiness and Surprise) for all of the children 
compared to the negative emotion (Sadness, Anger, Dis gust, Fear, and Contempt). The result from the micro - 
expression (facial expression) shows that 72.9% of the participant show positive emotion feedback when 
using the mobile application and only 27.1% of the participants show negative emotion feedback. 
This proves that the mobile application can give positive emotion to participants almost 73.0%. 


Facial Expression for Urban Children 
Happines 
Ss 


15 14 


10 Surprise 
12 
== Urban Children 


Contempt 





Figure 4. The facial expression results urban children 


3.4. Kort scale 

The Kort scale evaluation aims to identify the user experience whether the participant like it or not. 
The scale range is from (+2, +1, 0 , -1, -2) where +2 means very likely, and -2 not likely. The evaluation 
includes the content, educational, appearances and organization of the e-learning application. Figure 5 shows 
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the result from the Kort scale rating where most of the participants rate +2 for content, educational, 
appearances and organization of the mobile application whereby having the percentage of 66.4%, 70.2%, 
80.8% and 71.2% respectively. 


GRAPH OF PERCENTAGE AND USER EXPERIENCE 
ACCORDING TO KORT SCALE 
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Figure 5. Graph of percentage and user experience according to kort scale rating 


3.5. Interview 

The expected outcomes in this research before its evaluation is to have user or children feel positive 
emotion toward leaming and school. This positive emotion will give the children positive experience 
in learning that could increase the interest to go to school among children. The evaluation of the mobile 
learning application was executed using 4 method where it is highly reliable and have been used by a lot 
of researchers. Table 9 shows that the result of all of the evaluation method shown that the mobile learning 
application could create a positive experience to user that leads to the production of positive emotion more 
than 70.0% and prove that the mobile leaming application was successful in creating the children interest 
in learning. Other than that, in order to confirm whether the e-learning application really could solve 
the problem in this research, which is to make the children interested in school, three extra interview question 
was asked to the participants. 


Table 9. Summary of the mobile learning application evaluation result 


Type of Result 
Evaluation Negative Emotion Positive Emotion (%) 
EEG Device 29 71 
Kort Scale 28 72 
Assessment score 5 95 
Facial Expression 27 73 


Table 10 show that the result of the three interviews. The result from the interview shows that 
90.4% of the participants feel positive after using the application. Next, 97.1% of the participants like to go 
to school when the application is implemented in their school. Lastly, 80.4% of the participants would like 
to recommend the application to their friends. 


Table 10. Result of interview question based on their experience 


Interview Question Response N Percentage 
Interview I Yes 94 90.38 
(Feel a positive emotion after use the application) No 10 9.62 
Interview 2: Yes 101 97.12 
(Interested in school after use the application) No 3 2.88 
Interview 3 Yes 93 89.42 
(Recommend the application to friends) No 11 10.58 
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Thus, all of the evaluation and interview show a positive result where it verifies the reliability 
of the research method created. This proves that the approaches used in the research method is capable 
in creating a positive emotion through mobile leaming application to the user. By applying the research 
method and implement it in the development of mobile learning application in the future, it is possible to put 
make children more interested to schoolfor learning. 


4. CONCLUSION 

The result of the evaluation of the positive emotion in children Mobile Learning Application 
from five method used prove that the mobile learning application (sains awal prasekolah) can trigger 
the positive emotion and positive experience in learning. The result of the evaluation were assessment score 
(95.7% with 9.545 standard deviations), EEG device (Jess than three minute is 70.7% with standard deviation 
of 13.087), facial expression (positive emotion to participants almost 73.0%), Kort scale (+2 for content, 
educational, appearances and organization with a total average of 72%) and interview (90.4% of 
the participants feel positive after using the application). Therefore, the Sains Awal PraSekolah mobile 
learning can be considered as the successful in projecting the positive emotion to children since all the result 
was above 70% for allof its evaluation. Meanwhile, this also proven that the method of the positive emotion 
was reliable and can be conducted with such significant result. 
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